Selection of Relevant and Non-Redundant Feature Subspaces for Co-training

On high dimensional data sets choosing subspaces randomly, as in RASCO (Random Subspace Method for Co-training, Wang et al. 2008) algorithm, may produce diverse but inaccurate classifiers for Co-training. In order to remedy this problem, we introduce two algorithms for selecting relevant and non-redundant feature subspaces for Co-training. First algorithm relevant random subspaces (Rel-RASCO) produces subspaces by means of drawing features proportional to their relevances measured by the mutual information between features and class labels. We also modify a successful feature selection algorithm, Minimum Redundancy Maximum Relevance (MRMR), to be used for feature subset selection and introduced Prob-MRMR feature subset selection scheme. Experiments on 5 datasets show that proposed algorithms outperform both RASCO and Co-training in terms of the accuracy achieved at the end of Co-training. Theoretical analysis of the proposed algorithms is also provided.